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Abstract 

The General Aviation Element of the Aviation Safety Program's Synthetic Vision Systems (SVS) 
Project is developing technology to eliminate low visibility induced General Aviation (GA) accidents. 
SVS displays present computer generated 3-dimensional imagery of the surrounding terrain on the 
Primary Flight Display (PFD) to greatly enhance pilot's situation awareness (SA), reducing or eliminating 
Controlled Flight into Terrain, as well as Low-Visibility Loss of Control accidents. SVS-conducted 
research is facilitating development of display concepts that provide the pilot with an unobstructed view 
of the outside terrain, regardless of weather conditions and time of day. A critical component of SVS 
displays is the appropriate presentation of terrain to the pilot. An experimental study has been conducted 
at NASA Langley Research Center (LaRC) to explore and quantify the relationship between the realism 
of the terrain presentation and resulting enhancements of pilot SA and pilot performance. Composed of 
complementary simulation and flight test efforts. Terrain Portrayal for Flead-Down Displays (TP-HDD) 
experiments will help researchers evaluate critical terrain portrayal concepts. The experimental effort is 
to provide data to enable design trades that optimize SVS applications, as well as develop requirements 
and recommendations to facilitate the certification process. This paper focuses on the experimental set-up 
and preliminary qualitative results of the TP-F1DD simulation experiment. In this experiment a fixed 
based flight simulator was equipped with various types of Head Down flight displays, ranging from 
conventional round dials (typical of most GA aircraft) to glass cockpit style PFD's. The variations of the 
PFD included an assortment of texturing and Digital Elevation Model (DEM) resolution combinations. A 
test matrix of 10 terrain display configurations (in addition to the baseline displays) were evaluated by 27 
pilots of various backgrounds and experience levels. Qualitative (questionnaires) and quantitative (pilot 
performance and physiological) data were collected during the experimental runs. 

Preliminary results indicate that all of the evaluation pilots favored SVS displays over standard 
gauges, in tenns of terrain awareness, SA, and perceived pilot performance. Among the terrain portrayal 
concepts tested, most pilots preferred the higher-resolution DEM. In addition, with minimal training, 
low-hour VFR evaluation pilots were able to negotiate a precision approach using SVS displays with a 
tunnel in the sky guidance concept. 

Introduction 

Synthetic Vision Systems (SVS) displays for General Aviation (GA) applications are designed to 
enhance pilot's situation awareness (SA), radically reducing the occurrence of Controlled Flight into 
Terrain (CFIT) accidents in addition to reducing or eliminating pilots’ spatial disorientation (SD), thus 
preventing low-visibility induced Aircraft Upset (LVAU) events. LVAU events without recovery might 
lead to a Low Visibility Loss of Control (LVLOC) accident. Using imagery derived from terrain, 
obstacle, and airport databases, SVS displays provide the pilot with an unobstructed view of the outside 
terrain, regardless of weather conditions and time of day. In addition, through the integration of advanced 
symbology (i.e. highway in the sky, velocity vectors, etc.), navigation performance is drastically 
improved, with no effective increase in pilot workload. 

The genesis of the Terrain Portrayal for Head Down Displays (TP-HDD) simulation experiment can 
be attributed to several factors. Actual requirements for terrain resolutions and texturing methods used to 
generate imagery on a PFD were previously undefined. In an attempt to utilize current technology 
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avionics platforms with limited computational power, industry SVS displays appear to be achieving 
substantial improvements in pilot performance over conventional displays. These new industry SVS 
displays utilize low to medium Digital Elevation Model (DEM) resolution databases (in the 30 arc-sec 
range) and simple terrain texturing concepts. At the time this paper was written, references were not 
available to document GA industry SVS performance improvements. In contrast, the RTCA (Special 
Committee 193 (SC-193) and EUROCAE Working Group 44 (WG-44)) have prescribed database 
resolution requirements intended to meet foreseeable aviation application that are much higher than 
current industry SVS applications. The RTCA committee recommends 3 arc-sec DEM for en route 
operations, 1 arc-sec DEM for terminal operations, and .3 arc-sec DEM for surface operations [1], Asa 
result, an inconsistency exists between GA industry SVS DEM data usage and the RTCA 
recommendations. In addition, there is a perv asive belief among various elements within the aviation 
community that high-resolution DEM data always provides significantly superior pilot SA and better pilot 
performance than lower-resolution DEM data. This belief is fueled, in part, by elaborate pictures of high- 
fidelity terrain portrayal SVS concepts. While higher fidelity terrain portrayals can be applied to SVS 
terrain displays, there is an inherent increase in SVS system complexity and cost. Previous SVS research 
has demonstrated that various concepts can work; however proportional benefits of high-fidelity SVS 
imagery were unknown prior to this effort. In-depth research to help define the terrain portrayal 
requirements for SVS displays is needed. The combined TP-HDD simulation and flight efforts explores 
these issues in order to attempt to generate critical data that will provide a basis of recommendations 
appropriate for all SVS applications, with a focus on GA. 

While previous studies have been conducted regarding the understanding of SA and SD, leading to 
some novel concepts, only relatively minor progress has been made towards measuring the effectiveness 
of these advanced flight display concepts [2-8], As an initial investigation, the SVS-GA team conducted 
a study that focused on determining the associated benefits of SVS displays towards reducing LVLOC 
and CFIT accidents for GA pilots [2], Results of that fixed-based simulator study demonstrated the 
effectiveness of generic SVS displays as compared to conventional GA flight decks in reducing pilot 
errors and thus improving pilot ability to control the aircraft during IMC. The current study builds upon 
results from reference 2 in addition to evaluation of multiple SVS display concepts. 

Objectives of the TP-HDD Experiment 

One of the primary challenges of SVS as a PFD is the presentation of appropriate information in a 
cost effective and computationally viable manner. The TP-HDD experiments focus specifically on the 
aspect of portraying terrain on a PFD. As previously mentioned, the TP-HDD study consists of two 
distinct efforts. The first (current study) focused on flight simulation while the next phase of the study is 
planned to be a flight experiment, employing the NASA LaRC Cessna 206 research aircraft. 

The objectives of the TP-HDD simulation experiment were to establish the relationship between 
terrain depiction fidelity and pilot's terrain SA, pilot’s performance (control and navigation), and 
prevention of LVLOC /LVAU incidents. The final goal of TP-HDD was to further establish the overall 
benefit of SVS for GA pilots. 

Upon completion of both simulation and flight experiment studies, results are expected to provide 
information to help establish the DEM resolution and texturing requirements for tactical HDDs, based on 
the phase of flight. Another anticipated outcome is to provide data to enable SVS design tradeoffs 
(performance vs. fidelity) and to develop integration of tactical HDDs with strategic terrain displays. In 
addition, this experiment directly provided support for the Federal Aviation Administration (FAA) 
Capstone-2 certification issues. The Capstone-2 program is endeavoring to develop and supply new and 
emerging avionics technologies to the commercial aircraft fleet in Juneau, Alaska, to reduce the extremely 
high-rate of aircraft accidents in that area. One technology being employed for Capstone-2 is SVS 
displays. 
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Method 

The experiment was conducted using a fixed based simulator equipped with a strategic (navigation) 
and a tactical (primary) flight display. Various terrain portrayal concepts were developed from 
combinations of DEM resolutions and texturing concepts. These displays were evaluated in the General 
Aviation Workstation (GAWS) facility at NASA Langley Research Center. Twenty-seven subject pilots 
participated in the TP-HDD simulation experiment. 

Display Concepts 

The display concepts evaluated can be grouped into three types. One type of display replicated 
conventional PFD's and was referred to as the blue-sky /brown ground (BSBG) concept. PFD's feature 
integrated information (i.e. airspeed, altitude, attitude) into one display (see Figure 1). 

Another type of display employed for this test was referred to as the baseline round dials (BRD) and 
is shown in figure 2. The round dial concept replicates instrumentation currently found in the vast 
majority of GA aircraft. 

The third type of display evaluated is referred to as the SVS display concept. SVS display concepts 
were identical to the conventional PFD, with the addition of the various SVS terrain portrayals in place of 
the blue-sky /brown-ground. Figure 3 is an example of one of these SVS display concepts, with the Photo 
Realistic (PR) texture and DEM = 1 arc-sec. 



Figure 1. SVS BSBG 
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Symbology 

Primary Flight Display 

The SVS PFD featured several advanced symbology elements, including air-data, orientation, and 
guidance information. Air-data information was presented by use of integrated airspeed and altitude tapes 
with a vertical speed indicator, including digital readouts of instantaneous values of indicated airspeed 
and altitude. A roll pointer with a sideslip wedge and magnetic heading digital read-out, and a pitch 
ladder portrayed heading and attitude orientation information. Additional symbology components 
displayed on the PFD were elements that characterized the velocity vector cluster. This cluster depicted 
current aircraft flight path angle and track angle, incorporating a non-quickened velocity vector with an 
acceleration-along-flight-path indicator (off the left finlet of the velocity vector marker) and sideslip flag 
(off the top finlet of the velocity vector marker). Guidance symbology was presented in the form of 
course deviation indicators showing glide slope and localizer deviation dots and a tunnel in the sky type 
of symbology (see Figure 4). The tunnel in the sky concept featured 420 feet wide by 320 feet tall, 
uniform green boxes depicting the current flight path for the approach scenario, providing lateral and 
vertical path guidance. Vertical and lateral path deviation indicators provided the pilot with information 
regarding proximity of the aircraft to the center of the tunnel. 
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Figure 4. Example of the Constant Color with Fish Net, DEM =30 arc-sec, with Tunnel 
Strategic Display 

SVS perspective displays are designed to proride pilots with terrain SA similar to and beyond the 
information provided by strategic terrain display concepts, such as the United Parcel Service Aviation 
Technologies (UPSAT) MX-20 multifunction display (MFD) (UPSAT MX -20). The presence of the 
MX-20 in this experiment provided the capability to evaluate the integrated information supplied by both 
the PFD and the MFD, as well as estimate the relative value of each type of display. The MX-20 was 
used in the Terrain Awareness Mode only (see Figure 5), and was located in the radio stack. On the MX- 
20 MFD, terrain awareness, route information, waypoints, and towers were portrayed. All display 
concepts, including both baselines, were evaluated in the presence of the MX-20 MFD. 



Figure 5. MX-20 MFD in Terrain Awareness Mode 


Terrain Databases 

Enclosing the Roanoke Regional Airport (KROA), the simulated area of operations, termed the ROA 
Sector, was located in a relatively mountainous area in Virginia (VA), and was defined by the boundaries 
of 37° 30'N, 79° 40'W, 37° 00'N, and 80° 40'W. The SVS display concepts developed were 
combinations of various Digital Elevation Model resolutions (from the ROA Sector) and terrain texturing 
concepts. 

Digital Elevation Model resolutions: DEM resolution defines the distance between elevation data 
points (post-spacing) for a given a given database. A low resolution. 30 arc-sec (900m/2953ft post- 
spacing) DEM, a medium resolution, 3 arc-second (90m/295ft post-spacing) DEM, and a high resolution, 

1 arc-sec (30m/98ft post-spacing) DEM, were investigated in this experiment. These three DEM 
resolutions were chosen with the intent of covering a broad range of viable DEM options. Specifically, 
the 30 arc-sec DEM is currently used by the industry and is a freely available database. The 1 arc-sec and 
3 arc-sec DEMs are part of the set of DEMs prescribed by the RTCA and form an upper bound for current 
consideration. 

DEM resolution specifies the post-spacing of the elevation data points. It should be noted that the 
higher resolution databases are larger in terms of overall data points for a given area of coverage, thus 
higher computational expenses are associated with manipulating these data. Since the databases for the 
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lower resolutions are less populated than the higher resolutions, substantial terrain features might 
potentially be excluded. The possibility of losing entire peaks in the lower resolutions, as well as detailed 
terrain relief, exists. In Figure 6, a "rounded'’ effect becomes apparent between the DEM=3 and the 
DEM=30 (refer to the area indicated by the arrows in Figure 6). For this example, the elevation-based 
generic texturing concept is employed. 



Figure 6. Difference between DEM = 3 (left) and DEM = 30 (right) 


Terrain-Texturing Concepts: Terrain-texturing refers to the method used to enhance elevation cues 
and position estimates of the DEM database. The three primary texturing concepts tested were Constant- 
Color (CC), Elevation-Based Generic (EBG), and Photo Realistic (PR). The constant -color texturing 
concept was developed to replicate a current industry concept that is in the process of Federal Aviation 
Administration (FAA) certification for the Capstone-2 program. This texturing concept represents a 
minimal SVS display concept requiring a nominal amount of computational resources for rendering 
available from current certified avionics platforms. The elevation-based texturing concept consists of 
different coloring bands that correspond to different absolute terrain elevation levels. Lower terrain levels 
are colored with darker colors, and higher terrain levels are assigned lighter colors. A certain shade of 
green was set to the field elevation, and the highest point was set to the highest terrain within 50nm of 
ROA, approximately 4,000 ft MSL. The photo-realistic texturing was derived from 4m satellite imagery 
data that was draped over the various DEM databases. For the CC and EBG terrain texturing display 
concepts, cultural features, such as roads, towers, and rivers, were presented on the display. Cultural and 
feature data were supplied directly through the photo-texture images on the PR displace concept. 

Fishnet (FN) Overlay Concept: In addition to primary terrain texturing concepts, a fishnet grid 
overlay was employed. The theory of the fishnet grid involves placing grids of known sizes within the 
synthetic scene to facilitate pilot's depth perception. The potential benefits of the fishnet grid used in this 
study are cues for depth perception, distance, and angular rates. In addition, this fishnet grid provided a 
constant geometric shape in the synthetic terrain scene. Flowever. the fishnet grid may introduce 
elements of confusion in terms of adding to display clutter, distracting the subject pilot, and being 
mistaken for the roads and rivers. These types of issues were investigated in this experiment. The 
spacing of the fishnet overlay was 500ft by 500ft, regardless of the DEM resolution. The grid was dual- 
color (gray/white), to compensate for different coloring of features within the terrain databases (i.e., 
lighter colors of populated areas for the PR texture). 

Test Matrix 

Combinations of DEM and texturing concepts (including the FN option) produced 18 different 
potential terrain portrayal concepts to evaluate. A preliminary look at these 1 8 display concepts revealed 
that the CC texture without a FN provided little evident information on terrain contouring. As a result, 
the CC texture without the FN was eliminated as a viable concept for terrain portrayal. For the remaining 
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15 concepts, a usability study was then conducted in order to down-select a smaller and more manageable 
set of candidate display combinations for more thorough investigation. While the formation of this set of 
candidate display concepts was based on subject pilot evaluations, the suggested collection of display 
concepts was broad enough to meet the objectives of the experiment. Subsequently, an adequate number 
of display -concepts (10) were retained for the core experiment (see Table 1). 


Table 1. TP-HDD Experiment Matrix (Legend: EBG=Elevation-Based Generic, PR=Photo 
Realistic, CC=Constant Color, FN=Fish Net) 
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These 10 SVS concepts were evaluated in addition to the specific baseline concept flight displays 
mentioned earlier. These two baseline flight displays, BSBG and BRD were split evenly between subject 
pilots with specific qualifications. All display concepts were randomized among the pilots, for each 
maneuver. 

One additional run was added to the above matrix for one specific texturing concept (Constant Color 
with Fish Net) and particular DEM (30 arc-sec), but without a tunnel during the approach maneuver. This 
trial was included in order to establish pilot performance and workload relationships regarding use of the 
tunnel in the sky guidance concept. 

Evaluation Pilots 

Twenty-seven evaluation pilots of various backgrounds and qualifications participated in the TP- 
HDD simulation and were grouped by certain characteristics. The first group of subjects consisted of 
fourteen low-time pilots with less than 400 hours and no instrument training beyond that required for the 
private pilot’s license rating. Six Instrument Rated pilots with various levels of experience, but less than 
1000 hours, made up the second set of subjects. The last group of evaluation pilots consisted of four 
professional test pilots from NASA and the FAA and three Juneau (Alaska) area commercial operators. 
The Juneau pilots are potential future Capstone II users. 

Simulation Equipment 

The GAWS facility platform is based on a modified Precision Flight Control PC-based Aviation 
Training Device (PCATD) Model PI-142 instrument procedure trainer (see Figure 7). The model PI-142 
uses hardware typical of a general aviation aircraft with left- and right-pilot seats. Modifications to the 
hardware included the addition of a 6” SVS-PFD and a UPSAT MX-20 MFD. 

Additionally, three different computers were employed to drive the system in GAWS. One 
computer, a Pentium-3 class PC, hosted the Initiative Computing (IC) Elite Electronic IFR Training 
Environment Version 6.2 software. This Elite software provided the aircraft dynamic responses to pilot 
control inputs, and control of the out-the-window weather, as well as data required to generate the 
research display imagery. Elite software can simulate various types of GA aircraft including a generic 
Cessna-172 model, which was used in this experiment. A Silicon Graphics ZxlO PC with a 3Dlab 
WildCat 4210 graphics card housed the SVS software employed to render the display presented on the 
SVS-PFD. This SVS display software included the underlying databases with the various DEM 
resolutions and texturing concepts, combined with the SVS symbology. The third computer, another 
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Silicon Graphics ZxlO PC with a 3Dlab WildCat 4210 graphics card, hosted the software that produced 
the out the window visual that was projected on the forward screen, to simulate the out the window view. 



Figure 7. The General Aviation Workstation Facility Configured for TP-FfDD 

A pilot-selectable Display Field of View (FOV) control was developed for the experiment. Field of 
View is defined as the horizontal FOV of the image presented on the display. The four FOV choices for 
this experiment were 22°, 30°, 60°, and 90°. A nullification factor (MF), which is the amount of angular 
compression to force an image to "fit” onto the display, is associated with each FOV. As the horizontal 
FOV increases, so does the MF. Fligher MFs make terrain features appear further away than they are in 
reality. Two control options were available to the subject pilots, one located on the left-hand side of the 
instrument panel and one positioned on the right horn of the yoke. Previous research data indicate that a 
single fixed FOV would significantly limit SVS effectiveness. While flying each of the evaluation tasks, 
the subject pilots were encouraged to scroll through the FOV options during each phase of flight, evaluate 
the options, and provide comments during this experiment, as well as supply a resulting FOV strategy at 
the completion of the experiment. 

Evaluation Tasks 

The evaluation tasks were developed to cover critical phases of flight. En route and approach phases 
of a flight were ranked as high priority for SVS applications by an earlier in-house study describing the 
SVS Concepts of Operation for GA [9], To add some additional sensation of realism and more 
representative levels of workload, a low/moderate level of turbulence was simulated throughout the 
experiment. 

En Route: 

The en route task required the evaluation pilot to maintain assigned heading, airspeed, and altitude 
values at different points during a 5-minute flight simulation. Each task began 19nm southwest of the 
Roanoke Regional Airport (ROA), with a heading of 140° and an airspeed of 100 KIAS. The high 
altitude en route task was initiated at 9,500 ft MSL (approximately 7,000ft AGL), while the low altitude 
en route maneuver began at 6,500 ft MSL (approximately 4,000 ft AGL). For both tasks, pilots were 
required to fly straight and level for approximately 2.5 minutes, maintaining heading, airspeed, and 
altitude. With the help of the strategic display to identify a fly-by waypoint, the evaluation pilots were 
asked to execute left turn, using 20 degrees of bank, to a heading of 050°, while simultaneously 
descending 1500 ft (over rising terrain). For this maneuver, part of the descent took place during the 90°- 
tum, while the rest of the descent was completed while maintaining the second target heading. The target 
level-off altitude for the high altitude task was 8,000 ft MSL (approximately 4,000 ft AGL), while the 
target altitude for the low altitude task was 5,000 ft MSL (approximately 1,000 ft AGL). At the starting 
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point of the maneuver, Visual Meteorological Conditions (VMC) was simulated. One minute into the 
flight a one-minute transition into Instrument Meteorological Conditions (IMC) was simulated by 
reduction of visibility on the out of window visual display to one statute mile. High altitude and low 
altitude maneuvers were chosen to examine the effectiveness of visual differences in the research terrain 
database at higher versus lower altitudes and to more comprehensively cover the GA operational 
envelope. 

Instrument Approach: 

The approach task consisted of a 6.5-minute flight simulation starting with a straight and level flight 
in VMC on a 30-degree localizer intercept course for the ILS 33 approach into ROA. The task started 12 
nm south of ROA at 2,600 ft MSL, with a speed of 90 KIAS. The subject pilots were tasked to fly a 
heading of 300° to join the localizer (approximately 10 nm from the threshold) and maintain 2,600ft until 
intercepting the glide slope at approximately 4.5 nm, then continue flying the approach to 200 ft AGL. In 
this case, out of window visibility was reduced from VMC to one statue mile within the first minute of 
the flight. In addition to moderate turbulence that decreased throughout the run, wind was simulated to be 
from 030° at 15 knots, decreasing to 5 knots on final. One objective of the instrument approach 
maneuver was to demonstrate that non-instrument rated pilots are able to fly to an acceptable level of 
precision, with minimal training, using an SVS PFD with tunnel symbology. 

Rare Event: 

The Rare Event task simulated a flight scenario with an incorrect altimeter setting. Effectively, the 
altitude tape indicated the incorrect (higher) altitude, which was different from the actual altitude 
portrayed by the terrain on the PFD. In addition, the altitude provided by the MX -20 also included the 
1,500 ft error. This task was administered at the very end of the data collection for each evaluation pilot 
and was designed to look like the low altitude en route task. The starting point was at the same position 
as the low altitude en route task, but actual altitude was 1500 ft lower. Consequently, the target level-off 
altitude for this maneuver was 500 ft below ground level. Display concepts, excluding baseline concepts, 
were randomized among pilots, repeating one of the display concepts the evaluation pilot had already 
flown. The purpose of the rare event was to provide a good experimental evaluation to determine whether 
or not the different SVS terrain concepts on the PFD provided enough improved terrain situation 
awareness to avoid CFIT accidents in a rare event. 

Operations 

The experiment was conducted within a 2.5 month-duration with no schedule interruptions. Each 
pilot participated in a day and a half of testing, consisting of approximately 35 trial runs, and 945 total 
trial runs for all pilots. Before the start of the experiment, each pilot received an extensive pilot briefing, 
as well as approximately one-hour of training time in the GAWS by a FAA certified flight instructor for 
instruments (CFII). The goal of these briefings and training was to familiarize each subject with the 
objectives of the experiment and educate the subjects on the salient features of the current SVS 
symbology set and the simulator functionality. The symbology set remained consistent across terrain 
databases and phases of flight, except for the tunnel-on and -off concepts. All high altitude trials were 
completed during one block of 1 1 runs; all low altitude trials were completed during the next block of 1 1 
runs; and all approaches were completed in the third block of 12 runs. As previously stated, the rare 
event was typically the last nm of the experiment for each subject pilot. 


Data Description and Analyses 

An exceptionally large amount of data has been collected for this experiment and detailed data 
analyses are in progress. The intent of this section is to describe the methodology of data collection, as 
well as describe the methodology used to analyze the data. A comprehensive report of the results of the 
experiment will be completed in the near future. 
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Quantitative Data 

Pilot Performance 

Pilot performance and control inputs for all flight conditions (68 parameters) were recorded at 30 
HZ. Both the en route and the approach maneuvers were manually subdivided into 7 different segments 
using custom-developed software. Figure 8 depicts the segmentation methodology specific to the 
approach maneuver, focusing on the MSL Altitude and roll angle data for one particular pilot. For the 
approach, the maneuver began with straight/level flight on a 30° localizer intercept course (segment 1), 
transitioning to the turn maneuver to capture the localizer (segment 2), the actual turn with a constant 
bank angle (segment 3), transitioning to straight/level flight on the localizer course (roll-out)(segment 4), 
straight/level flight tracking the localizer (segment 5), transitioning to descent and glide slope capture 
(segment 6), and the final descent portion of tracking localizer and glide slope (segment 7). 




Figure 8. Segmentation of a Scenario for the Approach Maneuver, Segment 1 - Segment 7 (S1-S7) 

One of the pertinent parameters in measuring pilot performance during an approach maneuver is the 
computed lateral and vertical path deviation (in feet). As a sample. Figure 9 shows the time history of 
these two parameters for the same evaluation pilot as the data shown in Figure 8, during the same run. 




Figure 9. Lateral and Vertical Path Deviation for a Typical Approach Maneuver 

Minimum, maximum, RMS, and standard deviations from target values of variables were computed 
for all pilots, scenarios and segments. This information then was cataloged into a spreadsheet which also 
included the run number, FOV, etc. The above information is then the basis for statistical post processing 
of the data using statistical analysis programs such as SPSS'®. 

Physiological Data 

The following types of physiological measurements were conducted: heart-rate (sensors bracketing 
the heart), skin temperature (a sensor on the right ring finger), and muscle response (sensors on the left 
fore-ann). Physiological data was collected for all pilots during the approach tasks and for the final four 
subjects during the rare event task. 


9 







Qualitative Data 

Pilot Comments 

Subject pilot comments were recorded real-time during each run, after each block, and during the 
final semi-structured interview, via the audio and video recordings. Over 216 hours of multi-media 
recordings were compiled. The multimedia recordings allow the researchers to determine what each 
evaluation pilot was experiencing, real-time, while dynamically working with each display concept. All 
evaluation pilot comments and remarks will be transcribed and analyzed. 

Post-Run Questionnaires 

A post-mn questionnaire was administered at the completion of each trial run. The post-run 
questionnaires consisted of three types of activities: Situation Awareness Rating Technique (SART), 
NASA Task Load Index (TLX), and Cooper-Harper rating scale. A SART is a situational awareness 
subjective measure. The NASA-TLX is used to analyze pilot’s mental workload. The Cooper-Harper 
rating scale was administered in an attempt to determine pilots' perception as to whether or not the 
different display concepts affected the handling characteristics of the simulated aircraft model, as well as 
provide a means to benchmark the simulation facility with the actual flight test aircraft. 

Block Questionnaires 

After each maneuver block, a block questionnaire was administered. This questionnaire consisted of 
the SA-Subjective WORkload Dominance (SWORD, see below) Technique, text questions regarding 
subject pilot FOV strategy, performance and terrain awareness ratings for each display concept, and 
general questions about the use of the primary and strategic flight displays. 

The SA-SWORD was developed by Vidulich and Hughes [8], Using the SA-SWORD, the subject 
pilots were asked to compare one display concept versus another, in a pair-wise comparison (across all 
display concepts), using a nine-point scale. These comparisons, for each pair of display concepts, 
indicate the subject pilot’s determination of the level of SA enhancement one display concept may 
provide over another display concept. 

Semi-Structured Final Interview 

At the completion of each subject pilot's experiment, an exit interv iew was conducted. The 
interviews focused on verbal and visual protocols to elicit participant responses. These interviews were 
flexible to draw upon comments from each facet of the experiment. 


Initial Results 

Samjtie Quantitative Results: 

Throughout the entire experiment, a trend emerged indicating that the subject pilots were 
experiencing various levels of stress when coming into close proximity with terrain during the rare event 
maneuver. As a result, the decision was made to collect physiological data on the remaining four 
evaluation pilots, during the rare event task. Data collection began after the evaluation pilot completed 
the turn to the target heading (about 3.2 minutes into the run). As an example of physiological data, 
Figure 10 illustrates the heart-rate data during the rare event scenario, averaged over the four subject 
pilots. Of interest on this particular graph is the amplification in heart rate, which occurred close to the 
end of the scenario. As the evaluation pilots flew closer to the terrain, their heart rate increased, 
suggesting that even though this was a simulation, conditions were realistic enough to induce some level 
of stress during this maneuver. 
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Figure 10. Average Heart-Rate During Rare Event Task 


Qualitative Results: 

The SA-SWORD data were calculated and a statistical ANOVA analysis was employed to determine 
if the findings were statistically significant. Specifically, terrain texture types (EBG, PR, and CC) were 
investigated, and the findings indicated that texture preferences were statistically significant 
(F(2,759)=126.136, p<001). Further analysis revealed that, according to a Student-Newman-Keuls 
(SNK) Post Floe test, no difference was detected in SA ratings between the PR and EBG texturing. 
However, a difference was evident between these two texturing types (PR and EBG) and the CC texture. 
The SA-SWORD data were also analyzed in reference to DEM resolution comparisons 
(F(2,759)=188.037, p<001), revealing a statistical significance. SNK Post Hoc test results demonstrated 
that three distinct groupings existed, DEM=1, 3, and 30. As DEM resolution decreased, so did the subject 
SA-SWORD rating. This finding supported subject pilot comments indicating that SA was improved 
using DEM = 1 versus DEM=3, as was the case between DEM = 3 versus DEM = 30. And, finally, the 
use of the FN overlay, versus no FN overlay, in terms of enhancing situation awareness was not 
statistically important. More in-depth statistical analyses are in progress. 

Table 2 illustrates a rank order of all display concepts (including each pilot's respective baseline), 
based on subject pilot preference. This particular pilot preference data was compiled during the final 
interview. Pilots were asked rank the order the display concepts they would prefer to have in their aircraft 
for an approach task. The table nomenclature is as follows: texture, Fish Net (if applicable), DEM. For 
example, EBGFN30 is the elevation-based generic texture, with a Fish Net, DEM = 30 arc-sec. The 
ranking structure is 1-11 with 1 being the highest score. 

Table 2. Subject Pilot's Rank Order During Approach 


Rank 


1 

EBG1 

7 

EBGFN30 

2 

PR1 

8 

PRFN30 

3 

PRFN1 

9 

CCFN1 

4 

EBGFN1 

10 

CCFN30 

5 

EBGFN3 

11 

Baseline 

6 

PRFN3 




This ranking indicates that during an approach, the most preferred terrain portrayal concept was 
EBG1. While CCFN30 was the least preferred texturing concept, by far, the evaluation pilots 
overwhelming preferred this concept to either the traditional gauges or the BSBG baseline. The above 
qualitative results may support a pervasive belief among various elements within the aviation community 
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that high-resolution DEM data always provides significantly superior pilot SA and better pilot 
performance than lower-resolution DEM data. However, the results of the quantitative data will show if 
comparable SA and pilot performance can be achieved with lower resolution concepts. 


Concluding Remarks 

One of the primary challenges of Synthetic Vision Systems as a Primary Flight Display is the 
presentation of appropriate terrain information, in a cost effective and computationally viable manner. An 
experimental capability was developed to address some of these terrain portrayal concept issues. While 
only initial results were discussed in this paper, an attempt was made to focus on the methodology of the 
experiment and analyses tool developed. Three blocks of pilot performance data (68 parameters) were 
collected for the 27 pilots, and run questionnaires were administered after every trial for each pilot, 
totaling 945 runs of collected data. Physiological data were recorded during the approach runs for each 
pilot, in addition to the four runs collected during the rare event tasks, creating 328 physiological data sets 
for analysis. 

Initial results strongly indicate that SVS displays with terrain depiction greatly enhance pilot 
situation and terrain awareness, and increase perceived pilot performance, without impacting pilot 
workload. In addition, with minimal training, low-time VFR pilots demonstrated the ability to execute 
precision approaches while using displays equipped with a tunnel in the sky guidance. More complete 
investigations are required before final conclusions are made on the interpretation of these data. 
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